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Abstract 

This article deals with the integrated use of online quizzes as a teaching and assessment tool in the general 
program of the subject Proyectos in the third course of Ingeniero Tecnico en Informatica de Gestion over five 
consecutive years. The research undertaken aimed to test quizzes effectiveness on student performance when 
used, not only as an isolated assessment tool, but also when integrated into a combined strategy, which 
support the overall programming of the subject. The results obtained during the five years of experimentation 
using online quizzes shows that such quizzes have a proven positive influence on students' academic 
performance. Furthermore, surveys conducted at the end of each course revealed the high value students 
accord to use of online quizzes in course instruction 
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1 INTRODUCTION 

The actual implementation of the European Higher Education Area in Spain has led to deep changes in official 
regulations and university course catalogues, and to the inclusion of interactive teaching and learning methods. 
With the application of these methods, student responsibility in the learning process, and the need to acquire 
both content knowledge as well as specific and generic professional-related skills, has increased notably 
(Secretaria General de Universidades, 2009). 

This paper presents the results of five consecutive years applying a combination of teaching strategies — 
discussion forums, group tutoring, collaborative learning, peer review and online quizzes (Salas-Morera, 
Cubero-Atienza, Redel-Macias, Arauzo-Azofra & Garcfa-Hernandez, 2012) — and focuses on analyzing the 
effectiveness of online quizzes. In general, the following secondary objectives must be achieved in order to 
involve students in these teaching-learning methods (Salas-Morera, Berral-Yeron, Serrano-Gomez & Martinez- 
Jimenez, 2009; Lan & Yan, 2009): a) students must be motivated to work regularly and to follow the correct 
sequence of activities in the course programme; b) it is essential to monitor working time of students and 
check that it is compatible with the number of credits assigned to the subject, c) the teacher should have 
sufficient time to properly attend to all student requirements regarding forum, correcting papers, tutoring, 
etc.; d) the students must receive quick feedback on each of the learning activities that they do, and e) the 
students should interact proactively with each other and with the teacher. 

In general, e-learning techniques involves a combination of teaching strategies (Lahwal, Amaimin & Al-Ajlan, 
2009), such as assessment or self-assessment quizzes, discussion forums, and tutoring, that provide tools for 
further and collaborative learning (Nevgi, Virtanen & Niemi, 2006). In this context, the teacher's role is critical 
(De Laat,& Lally, 2003), as the teacher must now move from being a mere transmitter of information (as in 
traditional educational systems) to becoming a content facilitator, technical consultant, and social animator —a 
more complex and laborious role (Jianhua & Yinjian, 2008). E-assessment has two key advantages: it 
substantially reduces teacher work time on assessments, and allows for rapid teacher feedback on student 
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learning outcomes (Whitelock, 2009). Therefore, depending on the objectives to be met, e-assessment is 
expected to be highly efficient. 

Furthermore, the use of online evaluation by means of a series of online quizzes (each one with little influence 
on the final evaluation) obtains better student academic results than using only standard exams as the learning 
assessment tool. Nevertheless, a set of precautions must be taken (Angus & Watson, 2009): a) online 
assessment methods should be effective substitutes for traditional ones; b) students should be at least 
indifferent to the use of these methods compared to traditional methods; and c) the online methods should 
achieve better results than traditional ones. Thus, it is highly beneficial to the learning process to use periodic 
quizzes as it compels students to work consistently and in accordance with other scheduled class activities 
(Savander-Ranne, Lunden & Kolari, 2008). 

With regard to implementing methodological changes required to adapt to the European Higher Education 
Area, this paper proposes the following objectives: 

• Analyze the effectiveness of using online quizzes as a teaching and assessment tool 

• Test the influence of quizzes on student academic performance 

• Analyze workload that quizzes suppose for students, and quiz compatibility with the other planned 
activities 

• Solicit student opinion regarding utility of questionnaires 


2 METHODS 


According to the ECTS system (Secretaria General de Universidades, 2009), the average student should be able 
to pass all subjects scheduled in an academic year with a total of between 1500 and 1600 working hours (in 
Spain) —equivalent to 25-30 hours per credit in a standard course of 60 credits. For this to be possible, it is 
crucial to establish a teaching methodology that integrates three key elements: a) is designed so that students 
acquire the knowledge and skills needed; b) student workload is consistent with subject objectives and credit 
allocation; and c) there is adequate information regarding methods, assessment criteria, and a detailed 
schedule of tasks. 


Therefore, when scheduling activities, the first issue is to make sure students can complete the required tasks 
in the total working hours assigned to the subject. To achieve this goal, the students completed a series of 
weekly surveys to determine the appropriateness of the allocation of hours for each activity and its consistency 
with the overall objectives of the subject. Specifically, the subject Proyectos is worth 4.5 ECTS credits, and thus 
requires activities totalling between 112 and 135 hours of student work time. Proyectos deals with project 
methodology, organization, and management. The subject aims to strengthen student abilities in: organizing 
and planning; analysis and synthesis; applying knowledge in practice; troubleshooting; decision-making; oral 
and written communication in a professional context; adapting to new situations; and managing technical 
information. It is expected that upon completing this subject, students will be able to organize the process for 
carrying out engineering projects, from managing technical documentation to implementing of solutions. 

The Moodle platform was used as a tool to support teaching, specifically in the area of discussion forums and 
periodic quizzes. The general structure of the subject is as follows: classroom lectures with the whole group; 
practical classes in computer labs in groups of 25 students; participation of students in online forum; forum 
activity feedback through group tutoring; periodic online quizzes online, and feedback on quizz results through 
group tutorials. Overall student assessment is broken down in the following manner: Final examination in 
theory and problems (40%); participation in forum (10%); computer classroom activities (12.5%); group 
tutorials (12.5%); and quizzes (25%). The following sections of this article will focus exclusively on the 
methodology used in online quizzes. 


The main purpose of online quizzes in Moodle is to encourage students to study regularly, and at a pace that is 
consistent with course requirements. The course is divided into three thematic blocks - TBI, TBII, and TBIII, 
respectively - and a total of six quizzes, evenly spread out during the semester. The topics covered in each quiz 
is as follows: 1) first half of TBI, 2) complete TBI block, 3) first half of TBII, 4) complete blocks TBI and TBII, 5) 
first half of TBIII and 6) complete blocks TBI , TBII y TBIII. Online quiz scheduling is designed to encourage 
students to work at the pace set by the teacher and in accordance with the other subject activities. The 
expectation is that by working at the pace set by the teacher, and in line with course objectives, students will 
display a more productive attitude in other subject activities. Each quiz is open for 4 days, and the total 
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estimated time for completion, including review of concepts, is 2 hours. There are several sets of questions for 
each quiz, which are shared by all teachers in various subjects in the area of engineering projects. These sets of 
questions vary in degree of difficulty, and are assigned randomly to students in different questionnaires. Table 
1 shows some examples of questions related to project planning (there is at least one correct answer). 


Question 

Answers 

Partial 

credit 

Answers 

% 

Correct 

Activities 

can be 

Those that the project manager choose 

-0.50 

6/25 

50.76% 

The smallest units of work in which the project can be broken 
down logically and it is possible to allocate resources and run¬ 
time 

0.17 

25/25 

The initial interviews of project definition 

0.17 

16/25 

All administrative tasks 

0.17 

17/25 

The wording of the document and binding 

0.17 

15/25 

The final installation in the system of the client company 

0.17 

18/25 

The meeting with the client company personnel to explain the 
operation of the application 

0.17 

16/25 

Those that are agreed between the project team and client 

-0.50 

5/25 

Critical 

paths 

and 

floats... 

They have the highest average time 

0.50 

12/23 

39.43% 

All critical paths are the same length 

-0.33 

18/23 

Delays in critical paths generate the delay of the entire project 

0.50 

20/23 

In activities with positive independent float, in no case is 
important to use all the free float 

-0.33 

3/23 


Table 1. Some of the questions used in quizzes and statistical data provided by Moodle 


After students have completed the two questionnaires for each block, the teacher gets from Moodle the 
statistics of students’ answers, analyzes the results (see last three columns of Table 1), and identifies the main 
weaknesses. This report contains the partial credit assigned to each answer, the number of students that chose 
each option, and the overall mark obtained. A 2-hour tutorial session is then scheduled with the entire group. 
During this session, concepts that produced the poorest quiz results are discussed with students in order to 
reinforce student understanding of them. Given the incremental nature of quizz content, students are required 
to periodically review concepts throughout the semester, which significantly improves students' mastery of 
subject. 

In addition to weekly surveys regarding student workload for each of the planned activities (which serve as 
feedback for activity scheduling for following academic year), other surveys regarding qualitative and 
quantitative aspect of the subject are completed by students at the end of each course. Moreover, we analysed 
the possible influence of questionnaires on the general student academic performance. The following section 
lists and discusses the results obtained. 


3 RESULTS 


3.1 Students' workload 


After every week of the semester, a survey is programmed in Moodle about the time the students spent 
working in each one of the activities of the subject: preparation and completion of questionnaires, forum 
activity, studying, preparation of papers and reports, and tutoring assistance. Table 2 shows the average results 
obtained over five years compared to the working time initially allocated. It shows how, in the students' 
opinion, the overall amount of work performed substantially exceeds the assigned according to the course 
credits (4.5 ECTS x 25-30 hours/credit = 112.5 to 135). In the new organization, 6 ECTS credits have been 
allotted for the subject (150 hours of student work), which should remedy this situation. 


Regarding the distribution of work over the semester (Figure 1), students spent a fairly uniform time in 
preparation for papers and quizzes during the whole semester, while personal study substantially increased 
when approaching the final exam. Although the forum is an assessable activity, it was the only activity in which 
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students worked less time than programmed over the years of study. In fact, although not in significant 
number, some students completely renounced participating in the forum, despite the fact that debates are 
meant to enrich the content of classes and tutoring sessions to foster more successful group work. Thus it is 
clear that students, at least in part, are not prepared for collaborative learning that requires more active 
participation than in traditional classes. 



Weekly 

average 

S.D. 

Total 

Initially 

allotted 

Preparation and completion of quizzes 

1.06 

0.47 

15.83 

12 

Forum 

0.53 

0.12 

8.02 

10 

Studying 

3.26 

1.26 

48.91 

35 

Papers 

0.96 

0.55 

14.44 

10 

Individual tutoring 

0.03 

0.06 

0.48 

1 

Classroom activities 

4.00 

0.00 

60.00 

60 

TOTAL 

9.85 

2.02 

147.68 

128 


Table 2. Average weekly workload of students in the five-years study. 

3.2 Quizzes 7 results 

Periodic quizzes via Moodle are intended to capture students' attention regarding the most relevant concepts 
of the subject and to link those concepts to other subject activities in order to increase participation and 
effectiveness of group tutorials and practical classes. As shown in Table 2, the average time spent on preparing 
and completing quizzes has been somewhat higher than projected in the teaching guide, which gives an idea of 
the importance that students give to this activity, which accounts for 25% of the final marks. Table 3 shows the 
overall results of the quizzes. The table shows irregular but consistently high participation and relatively high 
marks. 


9,00 


2 

I 



1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 


WEEKS 

■Individual tutoring HStudy ^Quizzes preparation and completion 

■Papers BForum activity 


Fig. 1. Average distribution of student work along the 15 weeks of the semester. 


A x 2 test was taken to ascertain quizzes' possible positive influence on students' final marks. The hypothesis (of 
the statistical test) was that students with marks of 6.5 or higher out of a maximum mark of 10, passed the 
theoretical part of the subject, and that there is a statistically significant relationship between the two 
variables, [x 2 (1, N = 336) = 3.19, p = 0.07], 
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3.3 Students' opinions 

Comprehensive surveys about the utility students attach to quizzes, or to participation in the forum activity, 
have been carried out at the end of each year under study to provide feedback for teaching guides to be used 
the following academic year. In response to the survey question regarding quizzes' usefulness in keeping up 
with subject content, students gave quizzes a rating between 4.06 and 4.20 out of 5. This, together with high 
levels of participation and high average grades, shows that quizzes has been very helpful. 


4 CONCLUSIONS 


The integration of quizzes with other instructional activities in a teaching strategy has been very favourable. 
Students highly appreciated the role of the quizzes that, with moderate effort, helped keep students up with 
the subject, strengthened their involvement in other activities, and had a very positive impact on academic 
outcomes. 


On the other hand, it has been proven that there is a statistically significant relationship between quizzes and 
other assessable activities, such as the theoretical part of the subject. Thus, although there are other activities 
in the subject, such as the forum, tutorials and group practices, overall academic success of those who made an 
average effort (more than 6.5 rating) on quizzes was very high. 

Given the results of using quizzes, and their relationship to the mark for the theoretical section of the subject, it 
is reasonable to consider eliminating the theoretical section and substituting it with quizzes. However, there is 
no certainty regarding the usefulness of quizzes for evaluating certain competencies that must be reinforced, 
and could perhaps be better assessed in an overall exam. On the other hand, quizzes have the added 
advantage of guiding students through the learning process with moderate and continual effort. This 
advantage would probably be lost if the quizzes had greater weight in the final mark. Likewise, increasing the 
weight of quizzes in the final mark would have a significant repercussion on student study hours, which are 
already very high in comparison with the number of study hours anticipated in the initial work plan. 


2006-2007 


86 

1 

2 

3 

4 

5 

6 

N 

80 

81 

77 

75 

74 

69 

MAX 

10 

8.5 

9.6 

8.8 

9.3 

10 

MIN 

1 

2.1 

3 

2.8 

2.3 

4 

MEDIAN 

7.5 

5.7 

7.6 

6 

8.6 

8 

MEAN 

7.4 

5.6 

7.3 

6.1 

7.6 

7.8 

PASSES 

69 

51 

73 

55 

59 

68 

S.D. 

2.0 

1.4 

1.5 

1.5 

2.2 

1.2 


2008-2009 


78 

1 

2 

3 

4 

5 

6 

N 

62 

60 

59 

63 

63 

63 

MAX 

8.4 

9.3 

9.7 

9.3 

9.7 

9 

MIN 

1 

2.4 

2.8 

1.5 

3.9 

3 

MEDIAN 

5.7 

6.3 

8 

8 

7.6 

7 

MEAN 

5.6 

6.0 

7.5 

7.0 

7.4 

7.0 

PASSES 

46 

41 

52 

47 

58 

61 

S.D. 

1.5 

1.8 

1.7 

2.3 

1.4 

1.2 


2010-2011 


38 

1 

2 

3 

4 

5 

6 

N 

31 

31 

29 

33 

23 

29 

MAX 

9.8 

8.7 

9.2 

8.7 

9.4 

9.6 

MIN 

3 

4.1 

4.0 

4.7 

6.6 

4.5 

MEDIAN 

7 

6.1 

6.7 

7.1 

8.1 

7.2 

MEAN 

6.9 

6.1 

6.6 

7.0 

8.1 

7.1 

PASSES 

26 

25 

26 

31 

23 

28 

S.D. 

1.8 

1.1 

1.3 

0.9 

0.8 

1.2 


2007-2008 


64 

1 

2 

3 

4 

5 

6 

N 

56 

62 

50 

53 

55 

59 

MAX 

8.3 

10 

9.6 

9.3 

9.3 

9 

MIN 

2.2 

2.6 

1.7 

3.3 

3.8 

2 

MEDIAN 

6.3 

6 

7.2 

6.1 

8.6 

7 

MEAN 

6.1 

5.9 

6.8 

6.1 

7.9 

7.1 

PASSES 

45 

45 

42 

41 

48 

56 

S.D. 

1.5 

1.6 

1.7 

1.4 

1.8 

1.6 


2009-2010 


70 

1 

2 

3 

4 

5 

6 

N 

65 

64 

59 

54 

56 

53 

MAX 

8.4 

9.6 

8.8 

9.8 

9.6 

9.9 

MIN 

2.2 

0.8 

2.7 

0.1 

4 

4 

MEDIAN 

5.7 

5.9 

6.5 

4.7 

8.2 

8 

MEAN 

5.5 

5.7 

6.5 

4.8 

7.8 

7.6 

PASSES 

39 

41 

51 

21 

50 

48 

S.D. 

1.4 

2.1 

1.3 

2.4 

1.5 

1.6 


Table 3. Quizzes results for the five-years study. 
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